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Evaluation and Selection of Academic Interns: 1967-1968 



In 1964 the American Council on Education initiated an internship 
program for Fellows in Academic Administration. The program was made 
possible by a grant from the Ford Foundation. The purpose of this pro- 
gram is to enlarge the number, and to improve the quality of persons 
wailable to fill key positions in academic administration. More specific 
operating objectives of the program are: (1) to identify qualified in- 

dividuals, (2) to select the most promising among those identified, and 
(3) to provide the selectees with experiences designed to develop their 
potential for effective performance in administrative positions directly 
related to academic areas. 

Following an initial planning and selection period the first 
group of fellows were on internship during the academic year 1965-66. 

At the same time, the Office of Research initiated a program of research 
on various aspects of the operational program in the implementation of 
its objectives. This research effort has focused on the first three 
years of the program and has produced three prior publications, Cox 
(1966) described the objectives and initial operating structure of the 
program. Astin (1966) presented the initial research results from the 
first year of the program; his report focused on the characteristics of 
participating institutions, the nominated candidates, the evaluation and 
selection process, and the selected fellows. Creager (1966) presented 
similar information for the second year of the program (1966-67), com- 
paring results with those reported for the first-year group and with 
information from control groups of non- part ic ipating faculty members and 

academic deans. 
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The present report presents and compares evaluation and selection 
information in the third year of the program (1967-68) with that from 
the first two years. In addition it presents, for the first time, infor- 
mation about the evaluation of candidates by the home institutions. The 
emphasis in this report continues the descriptive and normative aspects of the 
evaluation and selection, carrying the third year group through the 
process to the initial pos t- internship evaluation of selected interns. 

In a forthcoming report (Creager, 1971), the documentation of the evalua- 
tion and selection procedures will be carried one step further in order 
to present results of career follow-up studies of those who completed 
their internship, and to summarize the significance of the research 
program completed on the first three years of the program. Although 
certain changes in the operating program were introduced with the fourth 
year of the program (1968-69), and will be summarized in the forthcoming 
report, the research based on the first three years of the program was 
useful in supporting the decision to make operational changes. Moreover, 
the present and forthcoming reports provide a research-based documenta- 
tion of the experience obtained in the operational program. In spite of 
the relatively small numbers of persons in the program in any given year, 
the resulting information supplements the direct monitoring of the program 
accomplished by the operational personnel. 

Evaluation and Selection Procedures in the Operational Program 
With minor variations the evaluation and selection procedures used 
in the operational program were essentially the same in each of the first 
three years of the program. Presidents of the member institutions of the 
Council were asked to nominate a person considered to have outstanding 
promise for a career in academic administration. Each nominee submitted 

ERIC 
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a dossier consisting of a letter of recommendation from the nominating presi- 
dent* complete graduate and undergraduate transcripts* an application form, 
and an essay commenting on an article dealing with the selection of academic 
administrators. After initial screening for administrative reasons (complete- 
ness of information, meeting age requirements, etc.) the dossiers submitted by 
qualified nominees are evaluated by four-man teams, composed of outside 
consultants (usually university administrators) and professional staff of the 
Council. Each candidate thus received four dossier ratings, one from each 
judge, on a 5-point scale: Outstanding, Good, Acceptable, Doubtful, and 

Unacceptable. In the first and third years of the program, the essays were 
separately evaluated, but in the second year, the essays were included as part 
of the dossier. On the basis of the dossier ratings, varying numbers of 
candidates were selected for interviews* 

Each candidate passing the dossier evaluation screen is interviewed 
by two three-man teams composed of college presidents and members of the 
Executive Staff of the Council. Each interviewer rated the candidate's 
overall potential as well as a set of traits, selected for their presumed 
relevance to success in academic administration and for their presumed amena- 
bility to being judged in a brief interview* Each candidate was discussed by 
the members of the two interviewing teams and fellowships offered to those 
judged most promising as academic administrators* The numbers of qualified 
nominees and their disposition at the main stages of selection are presented 
for all three years in Table 1. Differences between numbers of qualified 
nominees and interviewees include a few withdrawals as well as those eliminated 
by the dossier screen* Some candidates withdrew after successfully passing the 
interview screen. 

Those selected as Fellows in Academic Administration typically 

0 
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Table 1 

Disposition of Nominees at Various Stages of the Program 



Disposition Group 


1965 


-66 


1966 


-67 


1967 


-68 


f 


% 


f 


% 


f 


% 


Total nominees 


128 


100.00 


218 


100.00 


161 


100.00 


Administrative eliminations and 














early withdrawals 


13 


10.16 


26 


11.92 


2 


1.24 


Qualified nominees (dossier rated) 


115 


89.84 


192 


88.08 


159 


98.75 


Interviewees 


113* 


88.28 


144 


66.05 


125 


77.63 


Those passing interview screen 


24 


18.75 


50 


22.93 


53 


32.91 


Those completing internship 


23 


17.96 


39 


17.88 


44 


27.32 



Dossier ratings were not used as a pre-screening device the first year. 
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spent the academic year at a "host" institution in the office of an 
experienced academic administrator, who usually served as the intern’s 
mentor. Interns were expected to do certain assigned readings in the 
broad field of academic administration and to prepare an analytical report 
of some significance in this field. He was then expected to return to 
his home institution. 

Background Characteristics of the Third-year Groups 

Several items of information from the application form were studied 
for winners, non-winners and for the total nominee group. The non-winners 
were further studied in terms of the stage of elimination, dossier or 
interview. Comparisons of the resulting information across groups and 
against the corresponding information from the second-year groups pro- 
vide useful information on some of the effects of evaluation and selection 
and how these changed. The results on background variables studied in 
both years are presented in Table 2. Data from the second-year group 
are quoted from a previous report (Creager, 1966) ; those from the third- 
year group are new. 

A rather consistent pattern of differences may be observed between 
the second-year and third-year nominees. On those characteristics most 
relevant to the aims of the program, the third-year nominees are of a 
higher quality. This strongly suggests that the nominating presidents 
are doing a better job of evaluation on their campuses than was the case 
in the second year. This is probably the result of a clearer concept 
of the aims of the program and the type of persons to be identified, 
an improvement resulting from experience with the first two years of the 
program. To the extent that this is true, it is favorable with respect 
to the changes in the operating program introduced in the fourth year 

0 
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where nearly all nominees become interns and the Council operations are 
focused on selection of Fellows, 

Obviously, if one starts with a better nominee group, the Council's 
evaluation and selection of Fellows has become a more difficult task. The 
differences between winners and non-winners, and those by type of elimina- 
tion, follow the same general pattern as previously observed but are 
systematically smaller in magnitude and of less statistical significance. 
This is exactly what one would expect if this nominee group was better 
than previous nominees on these background characteristics. Since infor- 
mation on these characteristics constitutes part of the dossier, we 
expect a greater increase in the quality of dossier eliminees than of 
interview eliminees and this is what is generally observed. At the inter- 
view stage, after the double screening at nomination and dossier evaluation 
stages, the appraisal is focused on a different set of variables. It is 
not an easy task in this third screening to make fine discriminations 
reliably. Some evidence will be presented in a later section that the 
interview judgments are, nevertheless, retaining much of their 
reliability in spite of the increased difficulty resulting from more 
selective pre-screening. 

This general preliminary picture of the third-year operations 
seems consistent with program objectives. However, it is conceivable 
that the trend could continue so that the program became locked-in at the 
nomination stage, with the loss of some individuals who would, in fact, 
become good academic administrators. Here the size of the program is a 
crucial point. Moreover, one may question whether all academic adminis- 
trators should necessarily have similar backgrounds. It is clear from 
the data in Table 2, as well as from previous reported data, that there 

o 
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still remains considerable individual variation among nominees and within the 
group of winners • Continued policy review and monitoring of nominee characteris- 
tics should prove useful to maintaining program objectives within the present 
size of the program. 

Evaluations at the Home Institution 

As a part of the dossier obtained on each nominee, the nominating 
presidents are asked to choose three persons familiar with the candidate to 
complete and submit a structured evaluation form. Typically the president 
completes one such form and asks two others to do so, such as a vice president, 
dean, or department chairman. The evaluation forms become a part of the 
dossier that is evaluated by the professional staff of the Council. The na- 
ture and validity of the information provided by these forms have not been 
previously reported. With the changes in the operational program discussed 
earlier, the importance of evaluations at the home institution becomes more 
critical. This section reports some characteristics of this information. 

The evaluation sheet consists primarily of a set of ratings on 
the 11 personality traits that are also evaluated in the interviews of 
those passing the dossier screen. In addition, ratings are obtained on 
Dependability, Personality, Acceptability to the Faculty as a Dean, 

Administrative Knowledge, and an Overall rating. Each form also seeks 
information on the rater-ratee relationship, including the number of years 
rater has known the subject. Out of 447 individual rating forms, 162 
raters knew the nominee for 1-4 years, 144 knew the nominee 5-8 years, 
and 120 knew him nine or more years. Twenty-one raters did not supply 
this information. The distributions for the Overall rating were essentially 
the same for the three groups defined by length of time rater knew the 
nominee and for the groups defined by relationship between rater andratee. 
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Dis tribut ions of Overall Home Institution Evaluation ratings are 
highly skewed, with most ratings at the top two levels of the five point 
scale. The result is higher mean ratings and lower variation than has 
usually been observed with the less skewed interview ratings. It should 
be noted that each nominee in the home evaluation has a unique set of 
raters and that the ratings are made in support of the nomination. This 
is in contrast to the interview ratings where the same group of judges 
observed all interviewees in a more nearly uniform situation. Moreover, 
the raters at the home institutions have observed the ratee over a period 
of time and in several situations, in contrast to the more highly specific 
and more structured interview situation. Nevertheless, the Home Evalua- 
tion ratings contain information useful for the dossier. An examination 
of the mean ratings and comparison of these with interview ratings clari- 
fies the different meanings and utilities of the two rating systems. 

The mean home evaluation ratings for each nominee were therefore computed 
and used in the correlational analyses on which most of the ensuing dis- 
cussion is based . 

Selected correlations among mean ratings for Home Evaluation (HE) 
scales, the Dossier, and Interview rating scales are presented in Table 3. 
The first column of Table 3 shows correlations of each HE scale with the 
HE Overall ratings; the second column shows corresponding correlations 
between specific and Overall ratings at the interview stage for scales 
common to the HE scales. Also for the common scales, both specific and 
Overall, the correlations between mean HE ratings and mean interview 
ratings are presented in column 3. The last column shows the correlations 
between the mean HE ratings for all HE scales and the mean Dossier rating. 
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NOTE: r nq = .16 for 158 nominees; r n = .18 for 124 interviewees 



One of the striking results is the low correlation (.09) between 
the Overall HE and the Overall Interview ratings. While at first sight 
this seems quite disappointing, further study of the related data 
clarifies the differences in what these two ratings measure and yields 
further insight into reasons for the low correlation. The HE Overall 
rating has high correlations with three of the four scales specific to 
home evaluation: 1 ependabili ty , Personality, and Acceptability as Dean. 

These items are more readily evaluated by raters at the home institution 
where there is more extensive opportunity for the rater to observe the 
ratee than in the interview. These scales also have the highest correla- 
tions among the HE scales with the mean Dossier ratings (column 4) , al- 
though they are lower than the correlations with HE Overall rating, in 
part because the Dossier rating contains other useful information (e.g., 
evidence of scholarly achievement and prior administrative experience) . 

Put another way, the unique information in the home evaluations is used 
in making the Dossier evaluation (r is .42 between HE Overall and Dossier 
rating) . 

Turning attention to the HE scales in common with interview scales, 
the pattern of correlations with the Overall HE rating is similar to that 
for the Interview scales with the Overall interview rating. The HE scale 
correlations are generally lower, a fact resulting from the difference in 
the content of the two Overall ratings and to the greater skewness of the 
HE scale distributions. The correlations between corresponding scales 
(column 3) are not very high and are, in fact, markedly lower for those 
scales more highly related to the Dossier rating. These correlations are, 
in fact, attentuated by the pre-screening of nominees on the dossier rating, 
as is the correlation between the two Overall ratings. 
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The home evaluations have clearly been meaningful and useful in 
providing specific and relevant information to the dossier even before 
the operational changes instituted in the fourth year of the program. 

With these changes it becomes even more important to ensure increased 
reliability of the home evaluations. Raters should be encouraged to make 
fuller use of the range of the scales, and to make finer discriminations 
between scale items when rating an individual. 

Nominee Evaluation: From Nomination Through Interview 

Although the foregoing discussions were concerned with evaluation 
of nominees, as focused on background characteristics and home evaluations, 
both included in the dossier, this section presents further information 
regarding the evaluation and selection process. In general previously 
reported relationships involving dossier and interview evaluations are 
confirmed for the third year of the program, as will be discussed after 
a brief digression to discuss a special study. 

Previous studies of the operational program indicated a high value 
placed by raters on "intelligence . n Because intelligence ratings were 
being made by different raters and in a variety of situations, it was 
thought that a standard test of high level intellectual ability might 
prove useful to the program. Accordingly, the third year nominees were 
asked to submit scores on the Miller Analogies Test, a well known and 
difficult test of verbal reasoning ability. These scores were used only 
for research purposes and were not included in the operational evaluation 
and selection process. Although 847> of the nominees submitted test scores, 
only 62% took the form (L) operational at that time. The results of this 
experimental testing are presented in Table 4 for nominees, winners, 
dossier eliminees, interview eliminees, and total eliminees. Intergroup 

ERIC 
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differences are quite significant and all are in the direction expected if 
intelligence (in the sense of verbal reasoning ability) is in fact being 
noticed and used, however indirectly, in the evaluation and selection process. 
Moreover, the mean score of 62 for nominees compares very well with means 
reported for a wide variety of educational, scientific, industrial, and 
management groups.'*' 

These findings indicate that verbal reasoning ability is a factor 
in the nominations, possibly by pre-selection to obtain a doctorate and to 
become a faculty member at the nominating institution, as well as in the 
evaluation of nominees at both dossier and interview stages. The test is. 
therefore, useful for characterizing groups and for monitoring the evaluation 
and selection process. Moreover, the size and pattern of group differences 
indicates that the test would be useful in early stages of evaluation and 
screening. There remains some question as to whether the costs of the testing 
and the administrative problems implied by the difficulty of obtaining recent 
and valid scores on all nominees justified by the potential gains in 

selection efficiency. One estimate of potential gain indicates that the 
validity of the dossier rating against the final interview rating (estimated 
as .50 in the full third year nominee group) might increase to .56 by addition 
of the MAT score. This could be a useful gain, especially where the dossier 
is used in prescreening nominees for interviews. It may be more practical 
to encourage evaluators at the home institutions to consider such information 
when, it is readily available at the nomination stage. 

" * 1 ” "" "" 

X A1 though these data are reported by the test publisher in the test 
manual, the individual studies were done by psychometricians at the using 
agencies. The author expresses his appreciation to Dr. Alexander Wesman , 

The Psychological Corporation, New York City, for cooperation in arranging 
for testing of the geographically dispersed nominee group. 
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The practice of using the mean dossier rating to screen out approxi- 
mately one- fifth of the nominees prior to the extensive interview process was 
introduced in the second year of the program and was done again in the third 
year of the program. The rationale for using the dossier screen was based on 
the substantial correlation between the mean dossier rating and the mean Over- 
all Interview rating, obtained on the unscreened nominees in the first year 
of the program (Astin, 1966). With the introduction of screening, correlations 
involving measurements after the screening can be computed directly only on 
the screened group, i.e., the interviewees. Such correlations are usually 
attentuated and are less directly relevant for validation of selection proce- 
dures . Corresponding correlations for the total nominee group may be estimated 
by statistical formulas (Guilford, 1950). Intercorrelations as obtained on 
the interviewees, and as estimated for the total nominee group are presented 
in Table 5. The obtained correlation of .35 between dossier ratings and 
Overall Interview rating compares well with the .39 previously reported for 
the second year group (Creager, 1966); the corresponding estimates for the 
total nominee groups are .50 and .57. 

The steady drop in the correlation between Dossier and Overall Inter- 
view ratings from year to year may or may not be a blessing. It may well mean 
that interviewers are doing a job less redundant on the dossier evaluation than 
was the case in former years. If the correlation were much higher, the justi- 
fication for the interview as permitting a "last look" at the candidate in a 
different setting would loose whatever s.ppeal that it has. On the other hand, 
as the correlation continues to drop, the justification for prescreening the 
dossier evaluation becomes weaker. It is still at a level which is useful in 
a multi-stage evaluation and selection process, especially when the administrative 
effort and interviewer time is taken into account. With current levels of 
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Table 5 



Nominee Evaluation Correlations 



3rd Year AAIP 3 



Variable 


1 


2 


3 


4 


N 


1. Dossier Rating (mean of 4) 


— 


.13 


.22 


.50 


159 


2. Essay Rating (mean of 2) 


.13 


— 


.18 


+ .07 


157 


3. MAT Score 


.19 


.15 


— 


.33 


134 


4. Overall Rating 

(Interview; mean of 6) 


.35 


- .14 d 


o 

4 1 


— 


125 b 



Obtained correlations below diagonal; those estimated for Total 
Nominee group above diagonal, 

^Eligible Nominees prescreened on Dossier Evaluation. 

c 113 interviewees, took MAT. 

^This is an example, frequently observed in data obtained from selected 
pilots, where a low positive correlation computed in the total unselected 
population becomes negative when computed within the selected group. 
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validity and program size, continuation of the dossier screen is recommended 
with no change in the approximately one-fifth elimination rate* 

The mean es^ay rating (variable 2 in Table 5) is the average rating 
given by two readers of essays submitted by the nominees. The essay assign- 
ment was changed from a commentary on an article by Robert M. Hutchins, required 

in the first two years of the program, to writing a memorandum to a college 

2 

president on how to set up a student newspaper. As in the case of the first 
year of the program, the essays were evaluated separately; in the second year, 
essays were considered as a part of the dossier. No statistical justification 
has yet been found for the essay requirement in the evaluation and selection 
of interns. Whatever value it has remains to be demonstrated in view of the 
administrative effort to obtain and evaluate the essays. 

Regression analysis of the mean dossier rating confirms the previously 
reported results: Persons high on educational achievement as evidenced by 

attainment of the doctorate degree and undergraduate grades, and of scholarly 
attainment are more likely to obtain high ratings on the dossier. The cor- 
responding multiple correlation is about .50, not quite as high as in the 
previous year (.55). It should be recalled that the Overall Home Evaluation 
rating correlates .42 with the dossier rating; inclusion of the Home Evaluation 
information increases the multiple correlation to about .60. The dossier 

2 

The essay requirement: 

The administration of a newly established public university is 
being urged by the student government to authorize the publication of a stu- 
dent newspaper. You are the president^ assistant, and he has asked you to 
draw up a memorandum, giving due consideration to each of the following points. 

a) Controls to be exercised by the administration. 

b) Student editorial freedom and accountability. 

c) Faculty or staff involvement. 

d) Position regarding the outside public. 
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evaluation, then, consists in part of ascertaining that the nominee evidences 
scholarly achievement, and that the home institution says he has the personality 
and dependability in personal relationships, and the administrative knowledge 
that would make him acceptable as an academic dean. 

Analyses of the interview ratings are generally confirmatory of pre- 
viously reported results. For example, regression analysis of the Overall 
Interview ratings on the specific ratings shows that the correlation of the 
Intelligence rating is increased from .69 to .80 by addition of Ability in 
Personal Relations, Personal Appearance, Extent of Speech (with a negative 
weight) and with small positive contributions from Enthusiasm and Poise. More- 
over, the rank order of validities (column 2 of Table 3) of the specific scales 
versus the Overall Interview rating is the same as in each of the first two 
years of the program, although somewhat lower. These lower correlations are 
to be expected as the discrimination asked of the interviewers becomes more 
difficult . 



Summarizing the research results to this point, the following may be 



noted : 

1. Improvement in the quality of nominees. 

2. Continuation of and confirmation that the dossier evaluation 
emphasizes intelligence functioning in a scholarly way with 
the further information that home evaluation of potential 
ability for academic administration is noted and included in 
the dossier evaluation. 

3. Validation of the use of the dossier evaluation as a screening 
device for reducing the pool of nominees to be interviewed . 

4. Confirmation of the shift in emphasis from evaluation on intelli- 
gence as expressed in academic achievement in the dossier evalu- 
ation to emphasis on intelligence as expressed in direct personal 
confrontation, and skill in personal relations when "on the 
spot," in interview evaluation. 

While the foregoing discussion has been concerned with characterizing 
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nominees, winners, and eliminees at various stages of the evaluation and 
selection process, and has noted the stability of results from one year to 
the next, more significant information bearing on the impact of the program 
and possible improvements of evaluation and selection procedures are best 
obtained by followup studies ascertaining what has happened to the various 
groups who did or did not hold an internship. 

Immediate Postinternship Followup Studies 

The first opportunity to obtain followup information occurs at the 
end of the internship period. At this point, usually in May of the intern- 
ship year, the interns convene in a postinternship seminar. They are asked 
to complete a questionnaire at registration. In addition to eliciting informa- 
tion about the internship experience and current attitudes toward various 
academic matters, the intern is asked to supply names of four persons, including 
his mentor, acquainted with his work during the internship and from whom 
confidential evaluations of his performance as an intern might be obtained. 

The information resulting from the Intern Questionnaire and from his confiden- 
tial reports provides some insight into how the program looks to both interns 
and hosts. The very small numbers of highly selected subjects seriously limits 
detailed statistical evaluation. Nevertheless the highlights of the resulting 
information may be briefly summarized. 

The interns are practically unanimous in expressing the professional 
value of the experience. Nearly all agree that it broadened professional 
contacts, enabled them to learn new administrative techniques, and gave them 
experience in new problem areas and in dealing with various types of personnel. 
Moreover, many report that the experience deepened their understanding of the 
interrelations among administrative problems. A variety of other benefits are 
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mentioned spontaneously as specifications of "Other 11 benefits. These include 
insights into the home institution, learning about national problems with 
broadening and deepening insight into those involving higher education, seeing 
the academic picture as a whole, development of a personal philosophy of higher 
education and increased perspective, learning about administrative theory, 
and enabling the intern to know himself (or herself) more fully. 

The interns typically indicate the following experiences, duties, and 
contacts as most beneficial to them during the internship: 

a. The variety of contacts and participations, especially 
those involving visits to other institutions 

b. Discussions with mentors, administrators, staff, and 
faculty. 

c. Discussions about philosophy of education or of adminis- 
tration 

d. Contact with state boards and offices 

e. Attendance at regional and national meetings 

f. Opportunities to observe operations, personnel relations, 
planning, accreditation, and budgeting, 

g. Opportunity for reading, writing, and thinking. 

When asked how to make the experience more beneficial, some did not 
answer or merely indicated satisfaction with the program as experienced. 

Others, however, suggested more contacts of the kind described as beneficial, 
more involvement and inclusion in meetings, affairs, and decisions, more 
direct contact with mentor or other person responsible for the intern, and 
more time to think. There is some disagreement among the interns as to 
whether the internship should be more structured, or less so, and in what ways. 

Papers prepared during internship included those required by the 
American Council on Education, those prepared for host institution, and those 
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prepared for the intern's own use. The major benefit of paper preparation 
was cited by a dozen of the interns: aid in focussing, deepening, and ex- 

pressing their thoughts. Three indicated that paper requirements stimulated 
them to seek out and develop some of the beneficial contacts mentioned above. 
Some, however, were frank to state that the papers were a chore, interferred 
with other more important activities, or were of peripheral value. 

Similar qualitative information about the program itself was obtained 
from the confidential raters of the interns in addition to their ratings of 
the specific intern. General satisfaction with, and even praise, for the pro- 
gram is commonly expressed by the raters at the host institutions, which 
benefit in two major ways: through the specific staff studies and other 

duties performed by the intern, and through the astute and constructive 
criticism an intern can make. Such criticism from an outside perspective can 
be stimulating in forcing examination of the status quo and the values of the 
host institution. Suggestions for improvement of the internship program 
generally reinforce those given by the interns. 

Excellent return rates on the Confidential Reports have been exper- 
ienced with nearly all reports being rendered by mentors and other adminis- 
trative personnel with whom the intern had working contact during the intern- 
ship, although the amount of such contact was somewhat variable. The ratings 
of each intern were averaged across raters. Mean ratings computed across 
interns are in the super ior- to- out s tanding range, as expected since they are 
highly selected. The raters would very probably have recommended the ratee 
for an internship if the raters had been participating in the selection 
process. Moreover, most raters (80%) would want the interns they rated to 
fill an appropriate vacancy on their campus. Some of the ratings were difficult 
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to render, especially those on relations with other specific types of personnel, 
both academic and non-ac idemic ; presumably only a few raters have an opportuni- 
ty to observe the intern in relation to all of the personnel categories in- 
volved. Some correlations between the remaining scales, the Overall Intern- 
ship rating, and the “Growth during Internship" rating, are presented for 
third year interns in Table 6, with corresponding values on the first year 
interns shown for comparison. Although instability in the correlations 
computed on small groups make it inadvisable to interpret individual correla- 
tions, there is some consistency and sense in their pattern. 

It was the hope that the mean ratings from the Confidential Reports 
could be used as interim criteria of the evaluation and selection process. 
However, in spite of the plausibility of the patterns of correlations internal 
to these ratings, validities for the dossier and interview ratings against the 
Intern ratings are generally low and positive, but not statistically significant. 
This is true even after attempts to correct the correlations for attenuation 
from selection on the dossier and interview variables. 

One explanation for this result is suggested by the different nature 
of scales on which ratings are made in preinternship evaluation from those 
in the postinternship evaluation. The former emphasize various traits and 
general behaviors of the individual while the latter emphasize actual perfor- 
mance and growth during the internship. The specific traits of initiative 
and persistence rated in the Confidential Reports are not specifically rated 
in the home evaluations or in the interview situation. The “Overall" scales 
have a different meaning and context. Further study of the Confidential 
Report ratings, especially their means and standard deviations, suggests that 
they are reliable in the sense of inter-rater agreement on a given intern, but 
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Table 6 

Correlations of Confidential Report Scales with CR 
Overall and Growth Ratings 



Rating Scale 


Correlation 
With Overall Rating 


Correlation 
With "Growth" Rating 


1st yr . 


3rd yr. 


1st yx 


. 3rd yr. 


Overall 


... 


... 


.74 


.90 


Growth during internship 


.74 


.90 


— 


— 


Administrative judgment 


.90 


.94 


.61 


.84 


Achievement on assignments 


.77 


.63 


.44 


.45 


Initiative 


.79 


.41 


.80 


.37 


Persistence 


.69 


.11 


.68 


.18 


Would have recommended for: 










internship 


.90 


.81 


.80 


.78 


vacancy on campus 


.80 


.75 


.76 


.74 


Would expect to perform as: 










Department chairman 


.27 


.42 


.13 


.34 


President /small college 


.81 


.55 


.70 


.45 


President/junior college 


.62 


.62 


.40 


.43 


President/large college 


.51 


.43 


.40 


.44 


Academic vice president 


.62 


.71 


.47 


.64 


Academic dean 


.56 


.54 


.41 


.46 


Dean of students 


.04 


.39 


-.10 


.35 


Fiscal or business officer 


.19 


.15 


-.03 


.13 


Chief development officer 


.19 


.40 


.13 


.31 


Distinguished professor 


.13 


.06 


.18 


.06 



Interns only. 



yr 

I o 
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not very reliable in the sense of mean ratings distinguishing among interns 
at different institutions. In any case the must crucial question is the 
longer range validity of these three summary evaluations (Dossier, Interview, 
and Immediate Post internship) for predicting career status as academic admin- 
istrators. In view of the demonstrated content validity and interrelationship 
between the Dossier and Interview Ratings, one would expect moderate validities 
against such career status , and that they would tend to be similar for the 
two evaluations. The lack of substantial relationship within the intern 
group between each of these two evaluations and the Postinternship rating 
means that the latter is on its own, i.e., the explanation suggested above 
would be confirmed if we found little external validity for the postinternship 
ratings, while finding such external validity would mean that something is 
happening in the internship situation that is helpful to the career status 
outcome, even though such information would not bear on the selection pro- 
cedures . 

The career status followup study of both former interns and of former 
candidates who did not complete an ACE internship from the first three years 
of the program is the subject of a forthcoming report (Creager, 1971) . That 
report is focussed on the validity of the program as a whole, with selection 
and program effects confounded, and on the validity of various individual 
measurements whether preinternship, postinternship, operational, or experi- 
mental. In this context the comparative validity of selection and postintern- 
ship operational measures within the group of interns will be examined and 
discussed in terms of impact of the internship experience. The report of 
the results of such a followup study represents the major aim and a culmina- 
tion of the research program in support of the operational program. 
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